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Introduction 



Don Glass, Ph.D. 

Director of Outcomes and Evaluation 
VSA arts 




During a rich period of rethinking evaluation methodology in the 1970s, David 
Hamilton wrote that curriculum evaluation is a “dynamic human enterprise that 
changes in response to the object of study.” 1 As curriculum, classrooms, and 
circumstances change, approaches, methods, and strategies are adapted to 
capture a better understanding of teaching and learning in complex classroom 
settings. Evaluators and educators can then use this knowledge to make better 
judgments of the merit and worth of opportunities to learn, as well as inform 
curricular and instructional decision making. 

\/SA arts is an organization situated at the dynamic intersection of arts, 
education, and disability. In our education program work, we intentionally focus 
on the arts as a learning strategy, particularly in inclusive educational settings. 
For practitioners, this means exploring Universal Design for Learning (UDL) and 
Differentiated Instruction (Dl) as strategies to design curriculum and instructional 
opportunities that support all students. For evaluators of these inclusive settings, 
this means puzzling out how to understand the impact of these complex 
strategies in classrooms of students with diverse readiness, interests, and 
learning profiles. These are no simple tasks. 

To explore this issue and address a gap in the research literature, VSA arts 
convened evaluators and practitioners for The Contours of Inclusion: Arts 
Learning Outcomes and Evaluation Strategies, a research symposium in 
Baltimore, Maryland, on November 7, 2007. The symposium presentations 
highlighted the growing body of evaluation and research from the general arts 
education field. They also featured specific work that relates to students with 
disabilities. 

This publication is an edited collection of invited essays by symposium 
presenters and panelists. Similar to with the symposium presentations, 
contributors were asked to share evaluation strategies and data collection tools 
and explain the context of their development and use. 

The leading essay is by Dr. Dennie Palmer Wolf of the Annenberg Institute for 
School Reform at Brown University. Her essay frames Traci Molloy and Aamir 
Rodriguez’s documentation of their arts residency partnership between the 
Studio Museum of Harlem and Fannie Lou Hamer Freedom High School in the 
Bronx, New York. This documentation was developed out of an inquiry process 
that was part of the Center for Arts Education's Leadership in Practice program. 
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In her framing essay, Dr. Wolf begins by synthesizing insights drawn from special 
education (UDL), standards-based reform (equity and excellence), and arts 
education (imagination and innovation). She crafts these insights into a 
framework for collecting evidence around various dimensions of opportunity. Dr. 
Wolf then uses this framework to evaluate the educational opportunities that 
existed to support students in achieving learning outcomes from the Fannie Lou 
Hamer partnership. Dr. Wolf also provides an example of an observation tool that 
examines learning behaviors of various groupings of students. She concludes by 
advocating for a collaborative and reciprocal relationship between data gathering 
and capacity building for the participants in evaluation studies. 

The second essay, by Dr. Robert Horowitz of Teachers College of Columbia 
University, explains the history and context to the development of a precise and 
valid tool for observing learning behaviors. This essay provides a unique “behind- 
the-scenes” story of how a data collection tool was developed specifically for arts 
education. This process is informative for evaluators and practitioners. Dr. 
Horowitz demonstrates the value of systematic qualitative observation of arts 
learning for designing increasingly valid observation categories and descriptors 
for program evaluation and classroom assessment. The resulting observation 
tools can then be used to generate quantitative statistical data. The tool featured 
in the essay has been used in studies on the program work of ArtsConnection in 
New York City. 

The third essay is by Dr. Gail Burnaford of Florida Atlantic University. Dr. 
Burnaford argues for a “layered research” model that couples evaluation with 
teacher inquiry. Dr. Burnaford describes the inner layer as teacher-led action 
research. Data are collected using a documentation template from the Chicago 
Arts Partnerships in Education’s (CAPE) online Action Research Publication 
System (ARPS). She then shares the Effective Teaching (ET) survey that is used 
by the external evaluation team across sites. This essay provides a framework 
for understanding educational programs over multiple sites, which also uses data 
from the grounded inquiry and interests of teachers. In a sense, Dr. Burnaford is 
advocating for a collaborative, participatory form of evaluation that has aspects of 
professional development and capacity building at its core. Data is systematically 
collected across multiple sites, then managed and validated by program staff and 
external evaluators. 

The final section is an edited transcript of the symposium's final panel discussion 
featuring Dr. Donna M. Mertens, a professor at Gallaudet University and former 
president of the American Evaluation Association (AEA). Dr. Mertens explains 
the importance of the AEA program evaluation standards by connecting the 
reasoning behind the prioritization of evaluation standards to the concerns of 
teachers and the populations of students that they serve. A copy of the 
Standards for Program Evaluation is included with her essay. This final essay 
reminds us of the wealth of knowledge, guidance, and opportunities available 
through a professional evaluation organization. 



VS/4 arts 



Frameworks and Tools for Evaluating Arts in Education 



2 




For VSA arts, these essays touch on many of the emerging issues in, and needs 
for, using evaluation in the arts education field. On the technical side, several of 
the essays examine the process of developing or using precise and valid data 
collection instruments specific to the qualities of arts education and particular to 
the classrooms and students whom we serve. The essays provide us with some 
useful theoretical frameworks for thinking about learning opportunities, 
categorizing students, observing and documenting learning behaviors, and 
gathering meaningful and useful data locally and across multiple program sites. 

VSA arts invites you to join these senior evaluators in deepening and extending 
this work by developing your own data collection and analysis strategies that 
include data about students with disabilities. We also invite you to share your 
findings about the complexity of arts teaching and learning in inclusive settings 
with the arts education and evaluation fields. And finally, we would like to pose a 
few questions for consideration at future VSA arts symposia: 

• What forms of data collection can be flexible and universally designed tor 
all students? 

• How can we differentiate data collection and analysis to understand 
teaching and learning for students with specific disabilities? 

• How can we use evaluation strategies as professional development to 
better understand student learning and the effectiveness of the various 
instructional opportunities that we provide? 



1 Hamilton, D. (1976). Curriculum evaluation. London: Open Books Publishing. 
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Building and Evaluating 
“Freedom Machines”: 

When Is Arts Education a Setting 
for Equitable Learning? 




Dennie Palmer Wolf, Ph.D. 

Annenberg Institute for School Reform 



Introduction: “Freedom Machines” 

Ron Mace was an internationally recognized architect, product designer, and 
educator who went about his work and life using a wheelchair. Building on his 
experience, he evolved a design philosophy that challenged conventional 
thinking and provided the foundation for a more usable and inclusive world. Mace 
coined the term "universal design" to capture his idea that it was possible to 
design products, homes, and places of work, as well as the larger built 
environment, to be both pleasing and usable to the greatest extent possible by 
people of all ages, abilities, and life situations. The approach is responsible for 
the advent or ramps, showers with handles for rising and sitting, and soft-handled 
kitchen utensils that permit hands that are small, have arthritis, or have 
amputated fingers to cook. Bruce Hannah, a contemporary designer himself, 
refers to the designs and items that have resulted from this approach as 
“freedom machines .” 1 

Beyond the specific inventions, there are three great legacies of the Universal 
Design movement. The first is an image of human capacity as a continuum of 
users, all of whom share a set of common needs and aspirations, but who 
require different supports to express or actualize their capacities. The second 
legacy is related: it is the realization that the search for universal designs 
uncovers ways in which many people can become more mobile, agile, or 
capable. Thus, the curb cuts that assist wheelchair users also make walking 
easier for people with canes, parents pushing strollers, and two-year olds who 
are becoming independent walkers. Finally, curb cuts change pedestrian life, as 
anyone walking down the street is in the company of elders, small children, and 
parents, as well as individuals who conduct their lives using a cane, a walker, or 
a wheelchair. In a world informed by Universal Design, they can all stroll, hurry, 
do their errands, greet friends, and look into windows. 

Thus, in settings informed by the principles of Universal Design, it becomes clear 
that a disability occurs at the interface between an individual and a setting. This 
changes the notion of disability from a condition that a person has or is to a 
condition that a person experiences in a particular setting. Thus, millions of 
people would experience visual impairment in a world where prescription glasses 
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had not been invented — or were unaffordable. Correspondingly, with the 
invention of curb cuts and motorized wheelchairs, individuals who were once 
unable to travel outside the home are now pedestrians. 

The purpose of this essay is to explore a proposal that has long been at the heart 
of VSA arts. The arts, just like universally designed tools or physical 
environments, may be very powerful “freedom machines” capable of changing 
many preconceived notions about who can do what. To examine this idea in 
greater depth, I want to use the example of the curriculum developed in 
partnership by the Studio Museum in Harlem and the Fannie Lou Hamer 
Freedom High School. This could not be more apt: the museum’s core mission is 
to give voice to artists of the African Diaspora. The educators at Fannie Lou 
Hamer High School — like the activist for whom it is named — see education as a 
civil right. So their collaboration, like Ron Mace’s work on Universal Design, is 
also about “freedom machines.” 

This exploration has several parts. The first section examines how the initial 
ideas for Universal Design in the physical environment have evolved into designs 
for learning, with a particular emphasis on what the arts have to add. The second 
portion looks at how these principles apply to one of the units developed in the 
partnership between Fannie Lou Hamer and the Studio Museum. A closing 
section presents some of the implications for evaluating arts programs designed 
to involve a wide spectrum of students. The point is to explore what the arts have 
to teach us about Universal Design in education, particularly in an era where high 
standards are, at last, becoming the rule for all students. 

Building a Deeper Understanding of Universal Design for Learning 2 

In the years since the initial application of Universal Design to the built 
environment, educators and researchers have applied the concepts that Mace 
originated beyond the physical world. Additional understandings have come from 
the field of special education, from the equity and standards movement, and from 
arts education. 

Insights from Special Education: Ensuring Opportunities to Understand 

Research and development in Universal Design for learners with special needs 
have taught us a great deal about how education would need to change if 
teaching and learning were to become “barrier free.” Out of that work, a number 
of principles guiding Universal Design for Learning have emerged. For example, 
researchers at the Center for Applied Special Technologies argue for: 

• Multiple means of representation to give learners various ways of acquiring 
information and knowledge; 

• Multiple means of expression to provide learners alternatives for 
demonstrating what they know; and 
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• Multiple means of engagement to tap into learners' interests, offer 
appropriate challenges, and increase motivation . 3 

For example, as computer technology has evolved, it has turned the printed page 
into a flexible digital text that can be enlarged, illustrated, read aloud, translated 
on the spot, or enriched with embedded definitions and background information. 
What was once a single — and for some an exclusionary — display of meaning can 
now share meaning through multiple modes of representation. This has redefined 
the interaction between readers and texts, making it possible for more students 
to read, comprehend, and enjoy complex texts. Similarly, computer technology is 
making multiple means of expression possible: graphic organizers, time lines, 
and concept maps, allowing students to demonstrate their understanding using 
diverse forms of expression . 4 Finally, the addition of illustrations, animations, 
game-like structures of tiered challenges, and live video feed connecting 
students globally have opened up new avenues of engagement for many 
students. As with other instances of successful Universal Design, these digital 
learning environments turn out to have helped other populations as well, such as 
reluctant readers and English-language learners. Beyond that, computerized and 
enriched texts have led the way in thinking about the kinds of wide-ranging 
reading skills that all of us need in a world defined by the rapid flow of information 
in multiple formats. 

Insights from the Standards Movement: Ensuring Opportunities for 
Excellence 

Access to information and supports for engagement are foundations for equal 
opportunity to learn. But they are not sufficient. Research and reform efforts that 
focused on issues of equity, both pre-dating and fueled by the standards 
movement, identified an additional set of essentials for high-powered learning. 
What that movement exposed is how frequently the “keys to the academic 
kingdom” are reserved for a few already high-achieving students. The large 
majority of young people rarely encounter the high expectations, discussions of 
quality, high-demand assignments, or access to multiple sites for learning that 
are absolutely necessary for proficient, as opposed to basic, levels of 
achievement. Thus, the work on standards and equity can be thought of as 
extending the list of dimensions of inclusive learning, adding the imperative that 
all students must have sustained and meaningful access to: 

• Clear and high standards made public through examples, discussions of 
quality, focused response to student work, and tools such as rubrics; 

• Challenging assignments that provide for practice and eventual mastery of 
high standards. This means all students being involved in the critical phases 
and processes of learning (e.g., drafting, research, editing, critique, etc.); and 

• Supports for reaching high standards in a timely fashion (e g., peer 
coaching, tutoring, after-school programs, summer sessions, learning the 
skills for asking for help, etc.). 



\/SA arts 



Frameworks and Tools for Evaluating Arts in Education 



6 




Insights from Arts Education: Ensuring the Opportunity to Imagine 

Recently the discussion of educational outcomes has begun to stress the 
importance of students developing their creativity and their ability to work with 
others to innovate. In a global economy, consistently and quickly affected by new 
technologies, it is clear that in order to thrive, young people will have to be able 
to apply understanding to new problems in new contexts in imaginative ways. 
This suggests that all students, not a small elite who have been designated as 
“gifted,” have the right to be addressed as authors and inventors, not just as 
future service workers and clerks. To be contributing adults, they will need the 
skills to come up with new ideas, strategies, products, partnerships, and ways of 
doing business . 5 While much of this conversation tends to draw on science, 
technology, and business as its model, education in the arts and humanities may 
have a renewed role to play in this work . 6 

The work of writers and artists has a number of features that extend what we 
might want as principles informing the design of universal learning experiences. 
Most importantly, a number of these same principles inform the work of 
scientists, engineers, and entrepreneurs : 7 

• Originality: Artists and writers have as their job making visible, audible, and 
palpable what we would otherwise miss in the ordinary drill of daily life. To do 
so, they regularly use tools outside familiar language and logic to make their 
point: metaphors, symbols, allusions, connections, and exaggerations. By 
operating in a range of complementary ways, artists often provide the 
equivalent of glasses or rubber handles or ramps, placing within our reach 
what would ordinarily be beyond our powers. 

• Choices and Varieties of Excellence: The arts are domains in which 
excellence has many forms. There are not so many “right” answers as there 
are multiple, effective, powerful, or stunning ones. Think of all the ways there 
are to create a portrait. Or to write a love song. But to realize a powerful 
solution or new version means making choices about what to say and how to 
say it. 

• Stretch: The arts are about stretching beyond limits you once thought 
unattainable. (I know a musician who, in describing his experiences 
rehearsing with other skilled chamber musicians, claims, “They make me play 
better than I can.”) This apparent contradiction is about what happens at the 
intersection between high personal and interpersonal standards for 
performance. It is about imagining and pursuing a level of performance that 
goes beyond what is easily within your grasp. 

• Exchange and Response: In schools, student work typically has no real 
audience and response is typically restricted to grading or correcting work, a 
process that focuses largely on accuracy, leaving aside originality, impact or 
voice. But in the arts, work is done for the purpose of reaching and affecting 
an audience. The questions are: How has what I have created reached and 
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affected those who experience it?” and “What does this teach me about my 
current work and its possible evolution?” 

The basic proposition here is that when we assemble the insights from these 
three lines of work, we have an enlarged set of design principles that define 
learning environments in which many students could be fully productive. In these 
environments, three important and overlapping forms of access exist: access to 
understanding, excellence, and imagination. The argument is that all three are 
necessary for a very wide range of students, with different histories, experiences, 
physical capacities, and modes of processing information to perceive, 
understand, and create in powerful ways. Only when arts education lives up to 
these principles is it a “freedom machine.” Only then is it like a universally 
designed kitchen or car, putting feasts and travels within the reach of many 
learners. (See Table 1.) 

Evaluating Arts Education for Understanding, Excellence, and Imagination 

The Holocaust unit, featured on pages 16-23, was collaboratively developed by 
the Studio Museum in Harlem and the Fannie Lou Hamer Freedom High School. 
In this unit, teachers, teaching artists and students in a humanities class read 
Elie Wiesel’s memoir Night and students, in the role of artists, developed 
collages using phrases from the text to express the core messages of the work. 
The unit is designed to serve all ninth- and tenth-grade students, a number of 
whom struggle to pass district and state tests of proficiency. Some students have 
had the uneven K-8 schooling that many urban students experience; others have 
been in and out of formal schooling due to immigration and/or many changes of 
address. Others experience cognitive and emotional challenges. 

So suppose the school district called the question, recommending that given 
these students’ academic struggles, the school’s discretionary dollars should be 
spent on extra tutoring in literacy or mathematics. Suppose, in return, that the 
principal and teachers argued that the investment in arts learning was worth 
every penny since it provided high-quality conceptual learning for the full range of 
students at the high school? 8 What evidence could they, an outside evaluator, or 
a team of observers from the district collect and examine to back this claim? 

Often, in evaluations, researchers examine the outcomes for groups of students 
who do and do not receive a program or treatment. Although this approach can 
reveal whether or not a program had the expected effects, evaluation can do 
much more. It can also provide valuable information about how fully or well a 
program is being implemented and where it needs to be strengthened. In 
addition, it can help to build the capacity of the school or organization that is 
offering a program. But to accomplish these ends, an evaluation must go beyond 
the examination of outcomes to examining: 1) the opportunities that the program 
actually delivers (not just what it promises); 2) different episodes of learning; and 
3) who is learning. (See Table 2.) 
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Table 1: DIMENSIONS OF AN ENRICHED FRAMEWORK 
FOR OPPORTUNITY TO LEARN 

Opportunity to Understand: Dimensions Derived from Universal Designs for Learning 

Multiple Modes of Representation: Use of different materials, symbol systems, and displays of 
information, to give learners various ways of acquiring information and knowledge. 

Multiple Modes of Expression: Students have the opportunity to use different materials, symbol 
systems, and displays in order to demonstrate their understanding. 

Multiple Forms of Engagement: Includes diverse approaches to tap into learners' interests, 
offer appropriate challenges, and increase their motivation, persistence, and investment. 

Opportunity for Excellence: Dimensions Derived from Equity and Standards Movement 

Clear and Widely Shared Information: Directions, timelines, assignments, expectations, and 
rubrics are shared, printed, and discussed in language that has meaning for students. Students 
see examples from adult and student work that make the standards concrete and compelling. 

Sustained Access to Key Tools, Spaces, and Interactions: Students have access to the tools, 
spaces, and interactions they need to accomplish good work. This access is sustained and 
adequate for accomplishing good work. 

Challenging Assignments: All students have access to the big ideas and critical processes for 
doing good work. Processes are within the command of what students have had the opportunity 
to learn or are explicitly taught so that everyone can master them. All students participate in the 
critical processes for learning: developing ideas, planning, discussion, critique, and reflection. 

Supports for Learning: Students are supported, not penalized, for their evolving understanding. 
There are opportunities to ask questions, get extra help, take different approaches, revise work, 
or extend the hours and materials originally planned. 

Opportunity to Imagine: Dimensions Derived from Arts Education 

Originality: The expectation is that students will produce original work, choosing materials, 
images, and modes of expression in order to shape and communicate their ideas, insights, points 
of view, and feelings. To do so, there are the time, materials, and interactions that make it 
possible to do original thinking and to execute the work. Students are addressed as authors and 
artists, and helped to experiment, learn from others’ work, and select and refine their own works 
and performances. 

Choices and Varieties of Excellence: Students with diverse abilities and approaches can 
develop their own work to new levels and make contributions to the understanding and 
development of others' performances or work. Assignments and classroom interactions 
acknowledge the individuality of student messages, styles, and intents. 

Stretch: Teachers, teaching artists, and students agree to work to high standards, as individuals 
and as a collective. In addition, there is a mutual investment in the rising level of quality expected 
for final products or performances across a school year and over successive years. This is 
evident in critiques, discussions, and in the ways that student work is used to set new levels of 
expectation for individuals and for classes as a whole. 

Exchange and Response: Students have audiences for their work that go beyond the 
classroom. They have responses to their work that include learning about how their work affects 
other people. These opportunities might include a performance or exhibition for other students or 
their families that includes artists’ statements, audience feedback, and discussions with audience 
members. 
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Table 2: EVIDENCE OF OPPORTUNITY IN THE UNIT ON THE HOLOCAUST 

(pp. 16-23) 



Multiple Modes of 
Representation 


Students learn about the Holocaust through literature, discussion, film, and 
other visual images. They also discuss nonverbal forms of communication 
such as visual symbols. 


Multiple Modes of 
Expression 


Students have multiple opportunities to develop and demonstrate their 
understanding of the Holocaust and its meaning, which occur throughout the 
course of the unit: classroom discussion, collage, artists’ statements, and the 
text of what they say to the visitors they guide through that exhibition. 


Multiple Forms of 
Engagement 


The unit provides students with multiple and varied ways to engage with their 
own and others’ work: informal studio interactions and classroom discussions, 
independent work sessions, and formal and more public events, such as the 
trip to the gallery. 


Clear and Widely Shared 
Information about 
Expectations 


Students began the unit looking at the work of adult artists who use text as a 
part of their visual art. Throughout, they continue to look at adult and student 
work that make the standards concrete and compelling. They have the 
timeline and assignments for the unit. The criteria for good work are printed 
and discussed. 


Access to Tools, 
Techniques, Spaces, and 
Interactions 


Students all have copies of Night by Elie Wiesel. They have ample time on 
computers to experiment with typefaces and generate their selections. The art 
materials (paint, gel medium, and brushes) are of high quality. Students visit 
other gallery spaces and speak with curators. 


Challenging 

Assignments 


All students are expected to create a powerful graphic collage that combines 
words or phrases from Night with symbolic imagery in monochromatic hues 
and to write an artist's statement explaining the choices they have made as 
artists. These assignments demand that students think and work in both 
factual and symbolic ways. 


Supports for Learning 


Students were supported, not penalized, for their evolving understanding. For 
example, to help them grasp the enormity and consequences of the 
Holocaust, teachers and artists help students to connect to experiences in 
their own families and cultural histories, like the cruelties of the Trujillo 
dictatorship. 


Originality 


The unit required multiple forms of imagination: 1) entry into a different 
historical period and place; 2) empathy for cultural and ethnic groups different 
from students' own; 3) entry into the world Wiesel creates in Night in order to 
select images at the core of that work; and 4) the creation of new symbols 
evoking the experience of the Holocaust. 


Choice and Varieties of 
Excellence 


Students with diverse abilities and approaches developed their own texts and 
images. They made individual choices about what to emphasize and how to 
communicate that message through the choices they made as readers, 
typographers, and visual artists. The resulting student work illustrates how 
seriously students took their responsibility for individual choices. (See the 
examples of student work.) 


Stretch 


Classroom teachers and the teaching artist worked with students to create 
carefully considered and deeply felt images. The unit proceeded in layers, 
each layer designed to increase students' knowledge of the choices and 
options that they have in generating their works. 


Audience and Response 


In addition to classroom-based response and critique, the unit also contained 
experiences with real-world audiences, such as an exhibition in the school's 
gallery space. 
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Look at Opportunities as Well as Outcomes 



In a period in U.S. education where school quality and accountability have been 
defined solely in terms of outcomes (especially standardized test scores), the 
Holocaust unit reminds us of the importance of examining the underlying 
opportunities that students have to develop and achieve. Without that kind of 
inquiry, we have no way of understanding how a particular program or school 
contributes to students’ success (or struggles), and consequently, no map for 
locating what should be improved, strengthened, or widely used in other 
classrooms and schools striving for more inclusive forms of education. 

Therefore, an evaluator has to develop the tools and invest the time to find out, 
“What evidence is there that this program, or unit within this program, provides 
students with the different forms of opportunity that characterize a fully productive 
learning environment?” For example, in programs that use the arts as a major 
conduit for teaching and learning, with respect to the dimension of 
understanding, it would be important to capture if teachers and artists are using 
many media and approaches to work with the challenging concept of symbols. 
Those same observations should examine whether students have multiple 
opportunities, using different modes of expression, to demonstrate their growing 
understanding. With respect to the dimension of excellence, evaluators need to 
ask if there is clear evidence for challenge in the multiple assignments within the 
unit. In looking at access to imagination, an evaluator should ask whether or not 
students have the opportunity to envision new alternatives and to express their 
unique insights throughout the unit. Table 2 lists the kinds of evidence that an 
evaluator collected in looking at a similar unit on the Holocaust, where students 
were producing a poem, rather than a collage, using phrases taken from Night 9 

Look at Different Episodes of Learning 

The unit on the Holocaust involves students, teachers, and artists in many types 
of learning and interaction: reading, discussion, studio time, gallery visits, and as 
writing artists’ statements for a show of their own work in the school’s hallway 
galleries. Thus a full evaluation of this type of intensive teaching and learning 
requires looking closely at the learning that occurs during these different 
episodes: 

1 . Instances of formal instruction (e.g., classes with teachers and teaching 
artists, whether at school or in the museum’s galleries); 

2. Informal learning sparked by formal instruction (e.g., riding the subway back 
from the gallery, student collaborations during lunchtime as they finish their 
work for the unit); and 

3. Culminating performances or exhibitions of student work (e.g., the opening 
event for the show of Holocaust collages). 
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A strong program, well delivered, is one in which there is evidence of student 
engagement and learning across these different settings. If students are coming 
in early or returning to class at lunch to work on their projects, and if they are 
importing what they have learned in class into their informal collaborations, then 
there is evidence of an effective program. 

Look at the Experience of Different Groups of Students 

Often educational evaluations operate at the level of schools or districts, 
commenting on the quality of instruction, climate or achievement at that level of 
analysis. But research regularly points out that the variation in the quality of 
teaching and learning is frequently greater within a single school than it is across 
schools. And frequently the variability of experience is as great within a single 
classroom as it is across classrooms. Until evaluations address that variability, 
they are unlikely to help educators design and conduct learning that is fully 
inclusive. Therefore, it is vital that evaluations examine the learning experiences 
of different groups of students within classrooms and schools. How much access 
to understanding, excellence, and imagination do currently high and low 
achieving students have? What about English-language learners, or students 
with learning disabilities? 



Table 3: SAMPLE OF OBSERVATION OF THREE STUDENTS 



Minute 


Teacher 


Struggling Student 


Striving Student 


Thriving Student 


0-2 


Explains 
assignment to 
select words 
and phrases. 


Listens, fiddles. 


Takes out book; hunts 
for paper. 


Takes out book, turns to 
page, skims, offers 
examples of words, asks if 
they count. 


2-4 


Moves among 

individual 

students. 


Flips through pages 
Teacher notices, 
comes by, and talks 
through definition of 
hope, despair. 


Makes list of words, 
quickly gets about 12- 
15. 


Heads 2 columns: Hope, 
Despair. Makes list of 
words that express these 
emotions. 


4-6 


Returns to 
struggling 
student. 

Speaks to whole 
class: It does 
not have to be 
long, more 
important to get 
feeling & ideas. 


Finds several words 
with Teacher. 


Keeps going onto 
second side of paper. 


Reads over list, circling 
some that appear to be 
choices. 


6-8 


Speaking to 
struggling 
student: Got a 
lot, might 
choose the best 
for saying what 
you want. 


Continues to build a list 
of words, and then 
reads it over several 
times, erases some 
items. 

Reads the list again, 
goes back to the book. 


Continues. 


If we can’t find all we want, 
can we go to another part of 
the book? 


The observation continues. 
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Table 3 presents a sample of evaluation work from a similar project in which 
three students were selected for observation, each currently performing at 
different levels (struggling, striving, and thriving). 10 They were observed during a 
period when they were rereading Elie Wiesel’s Night to select words and phrases 
for their compositions. Looking at the first eight minutes of the observation 
session begins to show how each of the students, independent of his or her 
current level of achievement, is addressed as a thoughtful individual with the 
potential for creative work. Where records show consistently different patterns of 
activity, learning strategies, or the ability to generate original ideas: these are the 
places to go to work. 



Look in Ways That Build Capacity 11 

The work that teachers, artists, and students are doing in this and similar 
programs is demanding. Any evaluation process that presumes to measure or 
judge this work has an equal responsibility to build and support the capacity of all 
who are involved. This does not mean endorsing what happens blindly. Instead, 
it means using the luxury of an evaluation to establish a shared conversation 
about what is excellent, what is “at promise,” and what needs to improve. This 
has several implications: the questions pursued in the evaluation should be 
formulated between all the partners at the outset. Teachers, teaching artists, and 
students should be co-researchers, not just objects in the process. 12 Any episode 
of evaluation work should be designed to be useful to all the participants. For 
example, student interviews, while more time-consuming than surveys, can 
provide young people with the chance to formulate their thoughts, to share their 
work, and to learn to speak with adults about their experiences. In the same 
spirit, it is vital to talk with teachers and teaching artists about what the initial data 
appear to show, involving them in the interpretation, and asking for their insights 
into successes and frontiers for improving the work. In every sense, like the 
classrooms described earlier, an evaluation should be an opportunity to learn 
and to imagine how to overcome current barriers. To be worth the investment, an 
evaluation has to contribute to our understanding of the circumstances under 
which arts education lives up to, or falls short of, its promise as a “freedom 
machine.” 
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Case Example Documentation 

Traci Molloy 

The Studio Museum of Hariem 

Aamir Rodriguez 

Fannie Lou Hamer Freedom High School 




History of the Collaboration 

For the past seven years, Fannie Lou Hamer Freedom High School (FLH) and 
The Studio Museum in Harlem (SMH) have been working together with support 
from the Alternative School Consortium for Cultural Literacy, the Center for Arts 
Education, and Project Arts to integrate the arts into the curriculum as a way to 
reach learners. The collaboration has transformed teaching practice as well as 
impacted student learning and overall school culture. Each year, a teaching artist 
works collaboratively with the ninth- and tenth-grade humanities team to develop 
two hands-on art projects that reflect aspects of their curriculum. There are six 
core humanities educators who teach twelve courses, two per instructor. 

Students in the classes range from 14 tol 8 years of age. There is one self- 
contained classroom for students with special needs. Although there is a wide 
range of skills within the self-contained setting, the average instructional level in 
reading and writing is sixth grade. 

The partnership has allowed ninth- and tenth-grade students to have access to 
art instruction, from which they have traditionally been excluded. It provides 
direct support for students who would not otherwise receive arts instruction until 
the eleventh grade, and makes it possible for FLH youth to reach benchmarks 
outlined by the New York City Department of Education’s Blueprint for Arts 
Teaching and Learning.^ The curriculum incorporates basic art making 
techniques, vocabulary, art history, and visual literacy exercises. 

The Studio Museum in Harlem has also become a valuable resource for 
classroom teachers. Every ninth- and tenth-grade humanities student visits the 
museum yearly with the teaching artist and an accompanying humanities 
teacher. For many students, this is the first museum experience. The museum is 
committed to enhancing school curricula and encouraging innovative teaching 
practice, which includes increasing the level of interaction with works of art and 
engaging students in visual thinking strategies that increase aesthetic literacy 
and critical thinking skills. 
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The Unit of Study: The Holocaust 

During the Holocaust Unit, students used memoir as a tool for analyzing 
universal themes associated with human rights violations. Central to this study 
were investigations of how imagery and theme develop meaning within the text. 
Students studied two memoirs: Night by Elie Wiesel and Maus by Art 
Spiegelman. Students were asked to write a thematic literature paper that 
analyzed themes of hope and despair in both works. The broader understandings 
for the unit were: (1) Imagery is a tool that authors use to create empathy; (2) 
Imagery helps develop and strengthen the themes present in a literary work; (3) 
Every human being has certain natural rights; and (4) A human rights violation 
occurs when a person is denied his or her natural rights. 

Students worked with teaching artist Traci Molloy to create mixed-media 
monochromatic paintings/collages that incorporated text from Night, a quote they 
would later use in their papers. The objectives for the hands-on art component 
involved: 

• furthering students’ understanding of the Holocaust by visually interpreting 
a passage from Night; 

• working on new art skills, including typography, collage, and 
monochromatic painting; 

• viewing and discussing artwork made during the Holocaust, as well as 
work made by artists who combine text with a limited background palette; 
and 

• building on previous art concepts that dealt with symbolic imagery/color, 
focal point, and compositional design. 

After completing the mixed media collage, students were asked to write a one- 
page artist’s statement to accompany their artwork. In the written reflection, 
students explained their aesthetic choices, outlining their rationale for the 
monochromatic color, type (both font size and style), layout, and imagery. The 
statements were hung in partnership with each art piece in an all ninth- and 
tenth-grade gallery exhibition that showcased their Holocaust knowledge. Design 
ideas and exhibition layout were based on field trips to the Studio Museum in 
Harlem, as well as a field trip to the Museum of Jewish Heritage. 
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CURRICULUM MAP 

Big Ideas: 

• Artists observe and construct meanings about their environment. 

• The world is a source of inspiration for the artist. 

• Artwork can communicate a social/political agenda. 

• Artwork can challenge and provoke a viewer. 

• Art and social history/literature are intertwined, not separate. 



NYC 

Blueprint 


Residency Timeline 


Art Making 


Sketch 

symbolic 

design 


Paint background 
and explain 
rationale for hidden 
and/or revealed 
words 


Collage making 
connection 
between the 
passage from 
Night text and 
graphic image 


Exhibition of 
student work and 
statements 


Arts Literacy 


Discuss 
examples of 
paintings with 
text 

backgrounds 
(Ed Ruscha, 
Tim Rollins 
+K.O.S., and 
Kwang Young 
Chun) 


Typography and 
graphic design 
vocabulary 

Symbolic color 






Making 

Connections 


Read and 
highlight main 
theme/concepts 
from Night 








Community 

Resources 








Field trip to the 
Jewish Museum 
or the Flolocaust 
Museum, NYC 



Student Learning Outcomes: 

By the end of the humanities unit, students were able to: 



• Explain how imagery can develop or strengthen a theme within a text; 

• Deconstruct imagery within a text and explain its connection to the book’s 
theme or broader themes; 

• Construct their own imagery associated with a text; 

• Explain rights to which a human being is entitled; and 

• Explain what constitutes a human rights violation and give several examples. 
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By the end of the art unit, students were able to: 



• Develop a mixed-media collage that integrated text with symbolic 
color/imagery; 

• Demonstrate knowledge of monochromatic color by utilizing various tints and 
shades; 

• Demonstrate a basic knowledge of typography, particularly the significance of 
font and size selection; 

• Translate literary comprehension into visual imagery/comprehension; and 

• Write an artist’s statement that clearly reflected their compositional, aesthetic, 
and symbolic choices, as well as knowledge of new art vocabulary terms. 

Instructional Strategies: 

The teaching artist also collaborated with classroom teachers to differentiate the 
instruction for students with learning disabilities. The unit was designed so that it 
could be executed in a wide range of settings with minimal adaptation. The 
following strategies were used by classroom instructors: 



• Rather than asking students to immediately interpret a quote in visual form, 
students were asked to first use their literacy skills to analyze the quote and 
then brainstormed imagery associated with key words in the text. This helped 
motivate students who have processing and/or output issues. 

• Students were given additional time when needed to develop their ideas in a 
meaningful way. 

• To assist students with selecting appropriate type settings, students were 
given assistive technology (laptops) to work with, which allowed them to 
experiment with different fonts and text sizes, as well as make numerous 
copies of their final text for the collage. 

• Students with limited motor skills were assisted in cutting out their words, or 
were given the option to tear out their text if appropriate to the context of the 
piece. 

• For students who had trouble conceptualizing their final text composition, a 
photo was taken of the desired layout so they could then clear off their 
painting and use the photo as a visual reference, instead of having to 
reconstruct it from memory. 
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